Repository navigation
Conversation
Generator (corpus mutation + grammar-aware templates), runner (per-case per-runtime drivers with timeouts), classifier (crash/hang/divergence with the docs/parity G1-G12 known-kind filter), minimizer, and a CI-able seeded smoke mode over committed cases. Runtime trees are located via RUBY_LIB / TS_DIST / PY_SRC / ISCRS_BIN env vars; smoke skips unavailable runtimes so maps-CI (ruby + ts) stays green.
The committed smoke baseline was rebuilt through all four runtimes via rebuild-smoke.mjs (seed 20261120, 420-case pool, agree-or-known cases kept). Classifier fix: hangs and process-level crashes are never explained by known-kind signatures — only divergences and in-process crash buckets are filterable by the docs/parity kinds.
Campaign results (seed 20261117): 1007 agree / 268 known-kind / 325 findings over 6400 observations, clustering into 7 root causes: py parse hang (interscript-py#43), py escape ValueError crash (#44), ruby CaptureRef NameError crash (interscript-ruby#864), ruby alias- maybe silent miscompile (maps#292), py parse-boundary divergence (maps#293), unguarded self-run recursion (maps#294), py note-text parse corruption (maps#295). Minimized repros committed under findings/; generator escape literals corrected to the grammar-valid uXXX/U forms.
Comment on lines
+11
to
+31
| name: Seeded differential-fuzz smoke (ruby + ts) | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v7 | ||
| - name: Set up Ruby | ||
| uses: ruby/setup-ruby@v1 | ||
| with: | ||
| ruby-version: "3.4" | ||
| - name: Checkout interscript-ruby | ||
| run: git clone --depth=1 https://github.com/interscript/interscript-ruby.git ../interscript-ruby | ||
| - name: Install parser dependency | ||
| run: gem install parsanol | ||
| - uses: actions/setup-node@v7 | ||
| with: | ||
| node-version: "22" | ||
| - name: Checkout + build interscript-ts | ||
| run: | | ||
| git clone --depth=1 --branch main https://github.com/interscript/interscript-ts.git ../interscript-ts | ||
| cd ../interscript-ts && npm ci && npm run build | ||
| - name: Run seeded smoke corpus | ||
| run: node scripts/fuzz/smoke.mjs --runtimes ruby,ts --jobs 4 |
|
|
||
| if (opts.summary) { | ||
| const rows = findingList | ||
| .map((f) => `| ${f.case} | ${f.kinds.join(",")} | ${f.findings.map((x) => `${x.kind}[${x.runtime}]`).join(", ")} | ${f.findings[0].detail.slice(0, 120).replace(/\|/g, "\\|")} |`) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
WO20 differential fuzz harness (issue #284): generator, runner, classifier, minimizer, and the CI-able seeded smoke mode.
scripts/fuzz/generate.mjs— corpus mutation (truncate, byte-flip, line surgery, cross-map rule splices, WF-1/keyword/escape/unicode injection) + grammar-aware templates (nested any/capture/maybe, constraint combos, alias chains, duplicate froms, empty-string rules, bounded self-runs). Deterministic from a single seed.scripts/fuzz/run.mjs— per-case (isolated dir), per-runtime drivers with per-case timeouts; invalid UTF-8 normalized to U+FFFD identically for all four runtimes.scripts/fuzz/classify.mjs— crash / hang / divergence with the docs/parity G1-G12 known-kind signature filter; hangs and process-level crashes are never explained away.scripts/fuzz/minimize.mjs— delta-debugging shrink with persistence proof.scripts/fuzz/smoke.mjs+ committedsmoke-cases/(199 seeded cases, green on all four runtimes locally; ruby+ts in maps-CI via the newfuzz-smoke.yml).scripts/fuzz/TRIAGE.md— campaign report: 1600 cases / 6400 observations, 325 findings -> 7 filed issues (2 interscript-py, 1 interscript-ruby, 4 maps), with minimized repros underfindings/.Runtimes are located via env (
RUBY_LIB/TS_DIST/PY_SRC/ISCRS_BIN), defaulting to sibling checkouts; missing runtimes are skipped in smoke mode.Test plan
node scripts/fuzz/smoke.mjsgreen (all four runtimes locally; ruby+ts legs are what CI runs)scripts/fuzz/TRIAGE.md